o 

41 
"a 



CO 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 
G05B 19/04, 19/418, 15/00 



Al 



(11) International Publication Number: WO 99/12082 

(43) International Publication Date: 1 1 March 1999 (11 .03.99) 



(21) International Application Number: PCT/US98/ 18559 

(22) International FUing Date: 4 September 1998 (04.09.98) 



(30) Priority Data: 

60/057,505 
60/088,344 



4 September 1997 (04.09.97) US 
6 June 1998 (06.06.98) US 



(71) Applicant (for all designated States except US): DYNALOG, 

INC. [US/US]; Suite 125, 6001 N. Adams Road, Bloomneld 
Hills, MI 48304 (US). 

(72) Inventor; and 

(75) Inventor/Applicant (for US only): DESMET, Pierre [BE/US]; 
830 Highwood Drive, Bloomneld Hills, MI 48304 (US). 

(74) Agents: GIFFORD, Emest, I. et al.; Gifford, Krass, Groh, 
Sprinkle, Patmore, Anderson & Citkowski, P.C., Suite 400, 
280 N. Old Woodward, Birmingham, MI 48009 (US). 



(81) Designated States: AL, AM, AT, AU, AZ, BA, BB, BG, BR, 
BY, CA, CH, CN, CU, CZ, DE, DK, EE, ES, FI, GB, GE, 
GH, GM, HR, HU, ID, IL, IS, JP, KE, KG, KP, ICR, KZ r 
LC. LK, LR, LS, LT, LU, LV, MD, MG, MK, MN, MW, 
MX, NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, 
TJ, TM, TR, TT, UA, UG, US, UZ, VN, YU, ZW, ARIPO 
patent (GH, GM, KE, LS, MW, SD, SZ, UG, ZW), Eurasian 
patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), European 
patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, 
IE, IT, LU, MC, NL, PT, SE), OAPI patent (BF, BJ, CF, 
CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG). 



Published 

With international search report. 

Before the expiration of the time limit for amending the 
claims and to be republished in the event of the receipt of 
amendments. 



(54) Title: METHOD FOR CALIBRATION OF A ROBOT INSPECTION SYSTEM 




(57) Abstract 

The present invention relates generally to use of an industrial robot (Robot) and a measurement device (22) attached to the robot's 
flange (24) in order to complete measurements in a manner similar to a Coordinate Measuring Machine (CMM). In particular, the present 
invention relates to use of calibration techniques to maximize position accuracy and repeatability over time of an industrial robot as well 
as accurately locate the coordinate frame of the measurement device (22) (e.g. a camera (36), laser (34), or other device which produces 
output that can be expressed in 3D coordinates) that corresponds with the Tool Center Point (TCP) of the Robot-CMM System. 



BNSDOCID: <WO 9912082A 1_l_> 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spain 


LS 


Lesotho 


SI 


Slovenia 


AM 


Armenia 


FI 


Finland 


LT 


Lithuania 


SK 


Slovakia 


AT 


Austria 


FR 


France 


LU 


Luxembourg 


SN 


Senegal 


All 


Australia 


GA 


Gabon 


LV 


Latvia 


sz 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


BB 


Barbados 


GH 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 


GN 


Guinea 


MK 


The former Yugoslav 


TM 


Turkmenistan 


BF 


Burkina Faso 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


Bulgaria 


HU 


Hungary 


ML 


Mali 


TT 


Trinidad and Tobago 


BJ 


Benin 


IE 


Ireland 


MN 


Mongolia 


UA 


Ukraine 


BR 


Brazil 


IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Belarus 


IS 


Iceland 


MW 


Malawi 


us 


United States of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


UZ 


Uzbekistan 


CF 


Central African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CG 


Congo 


KE 


Kenya 


NL 


Netherlands 


YU 


Yugoslavia 


CH 


Switzerland 


KG 


Kyrgyzstan 


NO 


Norway 


ZW 


Zimbabwe 


CI 


Cote d'l voire 


KP 


Democratic People's 


NZ 


New Zealand 






CM 


Cameroon 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


PT 


Portugal 






cu 


Cuba 


KZ 


Kazakstan 


RO 


Romania 






cz 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






DE 


Germany 


LI 


Liechtenstein 


SD 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Estonia 


LR 


Liberia 


SG 


Singapore 







BNSDOCID: <WO 9912082A1 J_> 



WO 99/12082 



PCT/US98/I8559 



METHOD FOR CALIBRATION OF A ROBOT INSPECTION SYSTEM 

Cross-Reference to Related Application 

This application is based upon Provisional Application No. 60/057,505 filed 
on September 4, 1997 and on Provisional Application No. 60/088,344 filed on 
5 June 6, 1998. 

Background of the Invention 

I. Field of the Invention 

The present invention relates generally to robot calibration and more 
particularly to use of robot calibration to maximize absolute accuracy and 
10 repeatability of a robot-based measurement system. 

II. Description of the Related Art 

Within the general field of measurement/part inspection technology lies the 
technology related to devices known as coordinate measuring machines (CMMs). 
Manufacturing companies of any sort who desire to perform measurement of 

15 objects with a high level of accuracy have typically employed coordinate 
measurement machines to complete such tasks. A typical coordinate measuring 
machine consists of a three-axis machine with a touch probe mounted on a heavy, 
machined surface. The coordinate measurement machine generally further includes 
a controller which can be used to drive the touch probe until the touch probe makes 

20 contact with a surface on an object (generally referred to as "manual" teaching of 
the measurement positions). In the course of performing the measurement task, 
the controller displays and records the position of the touch probe at a plurality of 
locations on the object being measured. 

Recently, as a result of the expense and limitations upon speed of the 

25 measurement process inherent to coordinate measurement machine technology, end 
users have begun to investigate use of less expensive, fast-moving industrial robots 
to carry measurement devices in order to collect measurement data faster allowing 
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measurement of a larger number of objects. Furthermore, the fact that industrial 
robots are manufactured in large volumes alone insures that such "robot inspection 
systems" offer the opportunity to collect measurement data at a much lower cost 
per measured object than traditional coordinate measurement machine technology 
5 when used in applications which require less accuracy than traditional coordinate 
measurement machine technology. 

The essential element of a robot inspection system is the use of the robot 
♦ to position a measurement device in a location where a feature of interest (e.g. 
hole, edge, surface, etc.) is within the field of view or range of the measurement 

10 device itself. Typically, robot inspection systems employ a three dimensional 
sensor carried by the robot. Such three dimensional sensors may involve use of 
a variety of measurement technology including but not limited to cameras, lasers, 
combinations of cameras and lasers, and others. However, in applications such as 
surface inspection, a robot inspection system may employ devices which provide 

15 measurements in "less" than three dimensions. An example of one such known 
measurement device is use of the Perceptron Contour Sensor for two-dimensional 
measurement of objects. The Perceptron contour sensor comprises a sensor and 
illumination source positioned at an angle relative to one another within a housing. 
The illumination source, typically a laser light source, generates a plane of laser 

20 light normal to the light source, such as a prism or by other means. The plane of 
light from the light source produces a line on the surface of the object. The 
sensor, typically a Charge Coupled Device (CCD) camera, views the surface of the 
object at an angle relative to the plane of light generated by the laser light source. 
Optical filters are then utilized so that the CCD camera is sensitive to the particular 

25 wavelengths generated by the laser. Because the camera is viewing the surface of 
the object at an angle relative to the laser, the contour of the surface of the object 
results in a shaped line which is measured by the camera. These laser and camera 
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systems are also utilized to measure the contours of the surfaces or measure gaps 
between two adjacent objects, such as components of a vehicle. 

Irrespective of the type of measurement device employed by the robot 
inspection system, the principal benefits of using robots when attempting to 
5 perform measurement tasks are that: (1) industrial robots are manufactured in large 
volumes, (2) thus can be produced at a potentially lower cost per measurement 
volume, (3) the systems typically involve "non-contact" measurement devices, (4) 
the operator may position the measurement device using six degrees of freedom 
rather than three degrees of freedom typically offered by coordinate measuring 

10 machines, (5) the movement between measurement positions is at a much higher 
rate of speed, and (6) modern industrial robots routinely achieve position 
repeatability of better than 0. 100 mm. 

The principal problem with using robot inspection systems lies in the fact 
that although industrial robots are designed and manufactured to be "repeatable" 

15 in a specific set of conditions, they typically are not manufactured to achieve 
"absolute accuracy" and, further, are not designed to produce constant repeatability 
over time under different sets of conditions. Repeatability is defined as the robots 
ability to return to the same positions in space when executing a particular robot 
program many times. In contrast, absolute accuracy is defined as the robot's 

20 ability to achieve a desired position in space using commanded coordinates, for 
example, in three dimensional space. Historically, robot manufacturers have 
focused on improving robot repeatability as each robot program was initially taught 
by an operator driving the robot to desired locations in space and simply recording 
it, no matter what values/coordinates were assigned to that position by the robot 

25 controller - based on the controller's calculation of the values/coordinates for the 
position using "nominal" dimensions of the robot type. 

In applications such as robot inspection systems where the position of the robot is 
used to identify the location of features on the measured object (i.e. holes, edges, 



BNSOOCID: <WO 9912082A1_I_> 



WO 99/12082 



PCT/US98/18559 



surfaces), end users seek absolute accuracy of the robot position as well as 
repeatability. Factors such as tolerances in the manufacturing and assembly of 
robot components, deflection due to the effects of gravity, and other variations 
between an actual robot and the "nominal robot" used by the robot controller to 
5 calculate positions in three dimensional space, cause the industrial robot to achieve 
actual positions which vary from the desired positions (i.e. 998.135, 1002.723, 
999.052 rather than 1000, 1000, 1000 position in three dimensional space). 
As a result of these issues of "absolute accuracy", the current use of robot 
inspection systems has been limited to "process variation" (i.e. the identification 

10 of direction of trends in dimensions of measured objects rather than the coordinate 
measuring machine capability of identifying the magnitude of variation between a 
particular measured object and its nominal dimensions), assuming that 
environmental conditions such as ambient temperature do not change. In the case 
of change in ambient temperature, for example, the size of robot components will 

15 change according to the thermal coefficient of the material from which the 
component was formed, so even the task of "process variation is compromised in 
real- world applications. 

Attempts to solve the issue of absolute accuracy and process variation for 
robot inspection system applications have included: (1) use of a sensor in 

20 combination with an external measurement system, such as use of Steinbichler's 
COMET device together with Nothern Digital's Optotrack, to directly track the 
actual position achieved by the robot and its measurement devices in 
three-dimensional space, rather than relying on the position reported by the robot 
controller (effectively just using the robot just as a carrying device) at each robot 

25 position in the production measurement program; and (2) periodic "off-line" 
calibration of the of the robot inspection system using an external measurement 
system ("external calibration") such as Dynalog's DynaCal Robot Cell Calibration 
System or any other similar technique. The method utilizing an external 
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measurement device to identify the actual position of the measurement device 
mounted on the robot carries fundamental and practical problems including: (1) a 
relatively low level of accuracy which such external systems can achieve 
throughout the working envelope of a standard industrial robot; and (2) the high 
purchase price of such an external measurement system easily eliminates any cost 
savings and reliability benefits in the use of a robot rather than traditional 
coordinate measuring machine technology. 

An additional problem with the technique of using an external measurement 
system is the fact that the "tool center point" (TCP) of the measurement device 
mounted on the robot is not a "physical" one and therefore does not automatically 
correspond directly with the point of interest of the external measurement system. 
Furthermore, such individual "on-line" techniques typically do not identify the 
location of the fixture (the structure that holds the object to be measured) relative 
to the robot's base frame directly. Finally, techniques involving external 
calibration of individual components do not facilitate automatic, periodic 
re-calibration "on line" (i.e. without operator intervention) to identify changes in 
robot and tool center point of the measurement device (for example changes due 
to fluctuations in ambient temperature). 

Summary of the Invention 
An object of the present invention is to provide a method for 
calibrating the components of a robot inspection system in a single step using 
measurement data collected by the measurement device mounted on the robot. 
This method for calibration maximizes the positional accuracy and repeatability 
over time of the robot inspection system when functioning in a manner similar to 
a traditional coordinate measuring machine. The principal feature of the present 
invention is that the Measurement Device of the robot inspection system itself 
provides the data employed in the calibration process. As a result, the robot's Tool 
Center Point (TCP) can intrinsically be made to coincide with the true focal point 
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of the measurement device during the calibration process. Consequently, by 
employing "calibration targets" at known locations in the measurement envelope, 
the present invention automatic identification of "as built" parameters of all three 
components of the robot inspection system (i.e. the robot, the measurement device, 
5 and the location of the fixture holding the measured object relative to the robot* s 
base frame). 

The calibration method of the present invention not only includes the 
Identification aspect of the differences ("calibration parameters") between the 
actual robot, measurement device and fixture and their respective nominal 

10 parameters, it also includes the necessary compensation aspect. The preferred 
embodiment of the invention, as described below, allows the end-user to choose 
between at least two alternate methods for compensation, either to modify: (1) the 
original measurement robot program(s) to position the measurement device at the 
desired locations with extreme precision, or (2) to modify the measurement data 

15 collected after executing the original, "unfiltered," measurement robot program(s) 
to compensate for the effect of errors in robot position upon the measurement data. 

The calibration method of the present invention can be executed (i.e. both 
the Identification and Compensation aspects) periodically and automatically 
between production cycles of the robotic coordinate measuring machine. This 

20 aspect of the present invention provides the additional benefit of "on-line" 
verification of the overall position accuracy of the robot inspection system 
including identification of changes in the "as built" parameters. The Identification 
aspect of the present calibration method applies to differences between the "as 
built" parameters and the nominal ones; as well as to the "as built" parameters that 

25 change over time (due to environmental factors such as temperature or due to wear, 
etc.) and thus affect the robot inspection system's repeatability. 

More specifically, the object of the present invention is to provide an 
apparatus for calibrating the robot inspection system using a variety of 
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measurement devices, whether that device provides measurement in one dimension 
or more than one dimension where the measurement device acts as both production 
tool and calibration data acquisition device. In one of the preferred embodiments 
of the present invention, a laser camera yields 2-dimensional data and accurate 
5 measurement of surfaces and contours. This preferred embodiment utilizes a 
specific target which allows the determination of specific points related to that 
contour. A feature of this preferred embodiment is that the measurement of 
distances to one target taken from two perspectives will result in the determination 
of one unique point. This embodiment exemplifies the fact that the overall 
10 calibration method of the present invention is independent of the amount of degrees 
of freedom of the measurement device; for example, the use of a typical 3- 
dimensional measurement device is not required. ) 

Further features and advantages of the invention will be apparent from the 
description below. 
15 Brief Description of the Drawing s 

Figure 1 is a side view of a first preferred embodiment. 

Figure 2 is a side close up view of a first preferred embodiment. 

Figure 3 is a side cross-sectional view of target 38 taken along the axis 40. 

Figure 4 is a flow chart of a method of the preferred embodiment. 
20 Descriptio n of the Preferred Embodiments 

The method and devices of the present invention will be described in 
conjunction with the drawings. It will be understood, however, that the method 
of the present invention will have applications in other areas of manufacturing and 
that the description of the present invention with respect to the described 
25 embodiments is by way of example only. 

The primary embodiment of the current invention consists of a method 
which involves a series of steps which will be described in the following 
paragraphs. In the first step of the preferred embodiment one or more calibration 
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targets will be located within the measurement envelope. In the preferred 
embodiments, the location of these targets should, to the extent possible 
"encompass" the measurement envelope of the robot inspection system. Therefore, 
preferably the locations of the calibration targets should not be co-linear and 
5 further "encompass" a relatively large volume which surrounds the object to be 
measured. 

The location of the calibration targets, each one relative to the other (i.e. 
known and expressed in one common but arbitrary coordinate frame) although not 
required (except for the purposes described in the following paragraph) minimizes 

10 the number of robot positions required in the calibration robot program (thus 
reducing the number of calibration targets required and the number of different 
robot orientations at each of the calibration targets, and increasing the resulting 
accuracy of the robot inspection system following calibration). 

The location of the calibration targets needs to be known and expressed in 

15 the coordinate frame of the fixture only if the operator intends to perform off line 
programming (OLP) of the robot inspection system. OLP of the robot inspection 
system involves creation of a measurement robot program through CAD 
information and computer simulation. In these cases where OLP of the robot 
inspection system is intended, the collected measurement data that truly and 

20 directly represents the absolutely accuracy of the object being measured. 

The second step involves the creation of a calibration robot program. The 
program is designed to cause the robot and the attached measurement device to 
move into proximity of each one of the calibration targets, with a plurality of robot 
configurations in such a manner that all of the robot's axes are exercised to as great 

25 an extent as possible. At each position commanded in the calibration robot 
program, the measurement device must read and record a measurement of the 
location of the corresponding calibration target. In this preferred embodiment, the 
calibration robot program should contain a sufficient number of robot positions to 
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create at least as many measurement equations as calibration parameters that are 
identified (e.g. one 3D measurement produces three equations one 2D 
measurement produces two equations, etc.). 

The third step of the method of the preferred embodiment involves the 
execution of the calibration robot program created in the second step. The 
program is executed and the measurements of the respective calibration targets at 
each robot position commanded in the calibration robot program are recorded by 
the measurement device. The measurements are expressed (either directly or 
indirectly through standard six degree of freedom frame transformation ) in the 
coordinate frame of the measurement device, which corresponds with the robot's 
TCP. 

The fourth step of the preferred embodiment may take one of two paths. 
The first path being to calibrate for absolute accuracy of the robot inspection 
system, the second path being for calibration for repeatability over time of the 
robot coordinate measurement machine system. 

The first path of the fourth step is calibration for absolute accuracy of the 
robot inspection system. The software is executed to identify the robot inspection 
system parameters (i.e. calibration parameters of all three components of the robot 
inspection system including robot related parameters, tool center point (TCP) and 
fixture location) as necessary to maximize the absolute accuracy of the robot 
inspection system by minimizing the absolute accuracy residuals. The robot related 
parameters may include but are not limited to: Geometric parameters such as the 
Denavit-Hartenberg parameters and non-geometric parameters such as compliance. 
In this preferred embodiment a relatively large number of calibration parameters 
must be identified, therefore requiring the use of several calibration targets within 
the robot inspection system and/or the use of robot positions in the robot 
calibration program which involves a large number of different robot orientations 
around the calibration targets. 
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The alternate path through step four is executed when calibrating for 
repeatability over time of the robot inspection system. In this embodiment the 
calibration software is executed to identify the robot inspection system parameters 
(i.e. calibration parameters of all three components of the robot inspection system 
5 including robot related parameters, tool center point (TCP) and fixture location) 
as necessary to maximize the repeatability of robot positions by minimizing the 
difference between the residuals of the first time calibration and those of the 
present calibration. Only the calibration parameters that are susceptible to affect 
repeatability (i.e. change over time: link lengths are affected by temperature, axis 

10 offsets drift because of wear, etc.) need to be identified in this case, requiring less 
calibration targets and/or less robot orientations around each one of these targets. 

The fifth step of this preferred embodiment enables the operator to choose 
between at least two different options for using the parameter information 
identified in the calibration process to perform compensation for said parameter 

15 information. The potential options within step five include compensation for 
measurement robot program and compensation or measurement data. 

In the first option for step five of the preferred embodiment, the identified 
robot inspection system parameters are used to correct the original measurement 
robot program in an effort to drive the robot to the nominal locations originally 

20 intended. This option produces robot inspection system measurement data that do 
not require further modification or compensation (for example, the measurement 
data can be used "as is" for analysis of the production of parts). 

In the second option for step five of the preferred embodiment, 
compensation of measurement data, the measurement robot program is used in 

25 conjunction with the identified robot inspection system parameters to correct the 
collected measurement data for the difference between the originally intended, 
nominal robot positions, and the position actually achieved by a particular robot 
with "as built" differences from the nominal robot. 
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Application of this preferred embodiment on a particular measurement 
device (ie. the two dimensional contour sensor provided by Perceptron) is now 
described. With reference now to Figure 1, a calibration system 20 is shown. A 
measurement device 22 is mounted on the end of the robot arm 24. Both the 

5 measurement device 22 and the robot arm 24 are controlled by computer 26. The 
computer 26 also receives the image information from the measurement device 22. 
It should be apparent to one skilled in the art that separate computers could be used 
to operate the robot arm 24, the measurement device 22 and perform the 
calibrations as will be described below. 

0 The robot arm 24 generally comprises a plurality of hingeably and/or 

rotatably connected links 30 and is secured to a fixed base 32. The measurement 
device is preferably of the type including a light source 34, preferably a laser 34, 
but not limited to laser devices, generating a plane of light generally perpendicular 
to the laser 34. The measurement device 22 further includes a camera 36 mounted 

5 at an angle with respect to the laser 34 and the plane of light generated thereby. 
The camera 36 is preferably a CCD camera and is preferably "tuned" to be most 
sensitive to the wavelength of light generated by the laser 34. A preferred 
measurement device 22 of the type described is available commercially from 
Perceptron Inc. of Plymouth, Michigan. Generally the laser 34 projects a line onto 

0 a surface to be measured. The CCD camera 36, viewing the surface at an angle 
relative to the laser 34, measures the shape of the line on the surface and thus, the 
surface. The measurement device 22 is utilized to measure contours on surfaces, 
including gaps between assembled components, particularly vehicle components, 
but not limited to vehicle components. 

5 The calibration system 20 of the present invention utilizes the existing 

measurement device 22 which is mounted on the end of the robot arm 24, the 
existing computer program 26 which controls and receives information from the 
measurement device 22 and robot arm 24, and a target 38 of a known shape, 
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preferably a sphere 38, in order to calibrate the parameters of the robot arm. One 
skilled in the art will understand that the target need only be of a shape which has 
been previously defined. In this preferred embodiment the target 38 is a sphere of 
a known radius R. The target 38 is placed at an arbitrary location within the 

5 working envelope of the robot arm 24 and measurement device 22. The only 
limitation on the placement of the target 38 within the working envelope of the 
robot arm 24 is that the robot arm 24 be able to move the measurement device 22 
in a plurality (preferably more than ten) different locations or orientations in which 
it can still view the target 38. The number of measurements required in this 

10 preferred embodiment depends upon the number of parameters to be calibrated. 

The computer 26 is programmed to control the robot arm 24 to move the 
measurement device 22 to a plurality of orientations and locations. Preferably, 
these different locations and orientations fully exercise the different parameters of 
the robot 24 (i.e., different angles among each of the adjacent pair of links 30). 

15 At each measurement point, the laser 34 projects a line 40 onto the target 38 as can 
be seen in Figure 2. The CCD camera 36 stores an image of the line 40, shown 
in Figure 3, which appears to the CCD camera 36 as a discontinuous line 40 
including an arc 40a corresponding to a contour of the target 38. The computer 
26 calculates the center (ymi, zmi) of the arc 40a and the radius rmi. Since the 

20 line 40 may not have been drawn across the center of the target 38, the calculated 
radius of this arc (rmi) might be smaller than the target's 38 known radius R. For 
reference, the outer surface of the target 38a across its center is shown in phantom 
in Figure 3. 

A flow chart 50 illustrating the method of the present invention is shown 
25 in Figure 4. In the following discussion of this preferred embodiment, the steps 
referenced are found in the flow chart 50 and reference the preferred embodiment 
shown in Figures 1- 4 In step 52, the target 38 is placed within the robot working 
envelope as described above. The target 38 is then measured by the measurement 
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device 22 from multiple positions and orientations in step 54. In step 56, the 
two-dimensional center and radius of the arc from each measurement is calculated 
and stored in ymi, zmi and rmi expressed in the referenced frame of the 
measurement device 22. In step 58, the corresponding robot location for each of 
5 the calculated two-dimensional centers and radii (ymi, zmi, rmi) is stored. These 
robot locations (the end of the robot arm 24 expressed in the reference frame of the 
robot base 32) are expressed either directly in "joint" format Qli, j2i, j3i . . . jni, 
for n number of links), or in "cartesian" format with "configuration" in which case 
it needs to be converted to "joint" format through inverse kinematics, depending 

10 upon the specific robot interface. 

In step 60, the location of the center of the target 38 is calculated for each 
measurement location from the stored robot locations and stored as xti, yti,.zti, in 
the reference frame of the robot base 32. It should be recognized that although the 
first measurement may generate two possible centers, i.e. on either side of the 

15 measured line 40, this will quickly be resolved by the second or third 
measurement. The location of the center of the target 38, as calculated based 
upon each of the measurements, will be different. This is caused by the fact that 
the actual parameters of the robot 24 (such as the length of the links 30) differ 
from the ideal, the location of the measurement device relative to the end of the 

20 robot arm 24 differs from the ideal, and the lack of accuracy of the measurement 
device 22. 

In step 64, starting with the approximate values of the robot's parameters 
and assuming the accuracy of the measurement device 22, the center of the target 
38 is repeatedly recalculated, iteratively improving the approximation of the 
25 Denavit-Hartenberg (D-H) robot parameters including the lengths of the links 30 
and other parameters (e.g. mechanical flexibility, parallelogram geometry, etc.) 
until the deviation among the multiple calculations of the center of the target 38 
from each of the measurements cannot be further reduced. 



BNSDOCID: <WO 9912082A1_I_> 



WO 99/12082 



PCT/US98/18559 



14 



15 



20 



In step 66, the deviation among the multiple calculations of the center of the 
target 38 is tested. If the deviation cannot be reduced any further and the 
parameters are not experiencing further improvement (due to the margin of error 
of the measurement device 22 and possible parameters not considered), the robot 
24 is calibrated. If, in step 66 the deviation has been reduced by the most recent 
iteration of the calculation, then the calculation is repeated in step 64. All of the 
parameters of the robot (the arm 24, the location of the camera 36 relative to the 
robot arm 24 and the location of the target 38 relative to the robot base 32) can 
be resolved by minimizing the following equation: 



Since the initial nominal values for the robot parameters are sufficiently 
close to the actual ones to be eventually identified, the above equation can be 
linearized based on differential changes of the transformation matrices. This 
linearized problem can be solved iteratively by applying the known singular value 
decomposition technique until no further improvement in the identified parameters 
is obtained. This will yield the actual parameters for the robot 24, the location of 
the camera 36 relative to the end of the robot arm 24, and the location of the target 
38 relative to the robot base 32. 

The calibration system and method of the present invention require no 
expensive external sensors and requires only a simple (although accurately formed) 
target 38. It should be apparent that the target 38 could be of any shape, however, 
a sphere is preferred in this embodiment. Further, it should be recognized that 
more than one target could also be placed within the working envelope of the robot 
24 in order to more fully exercise different links 30 of the robot 24. Actually, in 
embodiments which identify the location of the fixture relative to the robot base 
frame, at least three different locations of such calibration targets are required and 
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need to be known accurately with respect to the coordinate frame of the fixture. 
Calibration of the robot arm can be done on line, even between the inspection of 
assemblies. Thus, if the temperature changes during the course of operation, the 
robot arm 24 can be recalibrated in order to insure accurate operation. 
5 The preferred embodiment as described above applies to use of a particular 

two dimensional measurement device. However it is clear that the use of other 
measurement devices - except for obvious items - will not deviate from the general 
procedure described above. 

Indeed, use of a typical x,y,z measurement device (versus the two 
10 dimensional device described above) will actually somewhat simplify the overall 
procedure. Specifically, Step 5 and Step 6 of the flow chart on Figure 4 can then 
be eliminated, since the measurement device directly yields the x,y,z location of 
the calibration target. In essence, the remainder of the procedure remains the 
same. 

15 Having described my invention, however, many modifications, including but not 
limited to measurement devices, targets, computers, and robots thereto will become 
apparent to those of skill in the art to which it pertains without deviation from the 
spirit of the invention as defined by the scope of the appended claims. 
I claim: 
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Claims 

1 1. A robot calibration system comprising: 

2 a robot having an outer end movable within a working envelope; 

3 means for controlling the outer end of said robot to a plurality of positions 

4 within said working envelope; 

5 a laser light source mounted on said outer end of said robot, said laser light 

6 source generating a plane of laser light normal to said laser light source; 

7 a camera mounted on said outer end of said robot arm at an angle relative 

8 to said laser light source; 

9 a target having a known shape positioned within said working envelope; 

1 0 said laser light source projecting said plane of laser light onto said target 

1 1 at each of said plurality of positions of said outer end of said robot arm, said 

12 camera generating a measurement of said shape of said target at each of said 

1 3 plurality of positions by viewing said projected laser light on said target; and 

1 4 means for calibrating said robot arm based upon said known shape of said 

15 target and said plurality of measurements by said camera of said projected laser 

1 6 light on said target. 

1 2. A robot calibration system comprising: 

2 a measurement device mounted on an outer end of a movable robot arm; 

3 a target having a predetermined contour; 

4 means for controlling the robot arm to move the measurement device to 

5 take a measurement of the contour of said target at each of a plurality of locations 

6 and orientations; and 

7 means for calibrating said robot arm based upon said known shape of said 

8 target and said plurality of measurements. 
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9 3. The robot calibration system of claim 2 wherein the measurement 

1 0 device comprises a camera and a light source mounted at an angle relative to each 

11 other. 



1 4. In a robot calibration system, where said robot operates within an 

2 operating envelop and includes an end effectuator with an associated vision system 

3 capable of determining distance and shape, the method of calibrating said robot 

4 comprising the steps of: 

5 storing a mathematical model of said robot; 

6 storing a calibration data structure having a predetermined number of 

7 elements, each of said data structure elements containing a nominal value, a actual 

8 value and a minimumization value; 

9 positioning a plurality of targets within said operating envelop; 
10 storing a mathematical model of said targets; 

H storing a sequence of robot operations, wherein said operations are 

12 functional to place said end effectuator and associated vision system in a 

13 predetermined location with respect to said targets, determine a distance and a 

14 shape of said targets and store the distance of said targets in said calibration data 

15 structure; 

16 executing said stored sequence of robot operations; 

17 calibrating said robot for absolute accuracy based on said calibration data 

18 structure. 

1 5. In the robot calibration system of claim 4, wherein said vision system 

2 comprises a laser and a plurality camera. 

1 6. In a robot calibration system, where said robot operates within an 

2 operating envelop and includes an end effectuator with an associated vision system 
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capable of determining distance and shape, the method of calibrating said robot 
comprising the steps of: 

storing a mathematical model of said robot, 

storing a calibration data structure having a predetermined number of 
elements, each of said data structure elements containing a nominal value, a actual 
value, and a minimumization value; 

positioning a plurality of targets within said operating envelop; 

storing a sequence of robot operations, wherein said operations are 
functional to place said end effectuator and associated vision system in a 
predetermined location with respect to said targets, determine a distance of said 
targets and store the distance of said targets in said calibration data structure; 

executing said stored sequence of robot operations; 

calibrating said robot for repeatability based on said calibration data 
structure. 

7. In the robot calibration system of claim 7, wherein said vision system 
comprises a laser and a plurality camera. 

8. A Robot Inspection System comprising: . 

An industrial robot having an outer end movable within a working 
envelope; 

means for controlling the robot to move said outer end of said robot to a 
plurality of positions within said working envelope; 

a measurement device, mounted on said outer end of said robot, the output 
of which (if only ID or 2D, for example) can be manipulated to provide 3D (or 
more degrees of freedom) measurement data; 

one or more targets positioned within said working envelope in the 
preferred embodiment of the location of such targets has been identified accurately 
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11 in a single coordinate frame, but this condition is not required for other 

12 embodiments; 

13 a fixture which holds a production part or assembly of production parts in 

14 a location within said working envelope of said robot in the preferred embodiment 

15 of the present invention said location of said targets are identified in the same 

16 measurement frame as said fixture, but this condition is not required for other 

17 embodiments; 

18 means for calibrating said robot, the location of the coordinate frame of the 

19 measurement device (TCP), and the fixture's location based upon measurements 

20 made by the measurement device at each target at each of a plurality of robot 

21 positions involving a plurality of configurations of the joint axes of the robot; 

22 means for using the calibration information for a particular robot, TCP of 

23 the measurement device, and fixture's location to insure that said robot and TCP 

24 actually reach a commanded position at which 3D measurements, for example, of 

25 a production part or assembly of production parts can be performed in a manner 

26 similar to a traditional CMM Device. 



1 9. The method for calibrating said robot, the measurement device, and 

2 fixture components of the robot inspection system of claim 1 , wherein said robot, 

3 the TCP of the measurement device, and the fixture's location are calibrated 

4 periodically between completion of production measurement tasks thereby 

5 eliminating the need for further compensation for effects of wear, temperature and 

6 other environmental factors that can affect the repeatability of the robot inspection 

7 system over time. 
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